Practical Open-Loop Optimistic Planning

We consider the problem of online planning in a Markov Decision Process when given only access to a generative model, restricted to open-loop policies - i.e. sequences of actions - and under budget constraint. In this setting, the Open-Loop Optimistic Planning (OLOP) algorithm enjoys good theoretical guarantees but is overly conservative in practice, as we show in numerical experiments. We propose a modified version of the algorithm with tighter upper-confidence bounds, KLOLOP, that leads to better practical performances while retaining the sample complexity bound. Finally, we propose an efficient implementation that significantly improves the time complexity of both algorithms

Semantic modelling of user interests based on cross-folksonomy analysis

The continued increase in Web usage, in particular participation in folksonomies, reveals a trend towards a more dynamic and interactive Web where individuals can organise and share resources. Tagging has emerged as the de-facto standard for the organisation of such resources, providing a versatile and reactive knowledge management mechanism that users find easy to use and understand. It is common nowadays for users to have multiple profiles in various folksonomies, thus distributing their tagging activities. In this paper, we present a method for the automatic consolidation of user profiles across two popular social networking sites, and subsequent semantic modelling of their interests utilising Wikipedia as a multi-domain model. We evaluate how much can be learned from such sites, and in which domains the knowledge acquired is focussed. Results show that far richer interest profiles can be generated for users when multiple tag-clouds are combine

Mineralization of an Axially Aligned Collagenous Matrix: A Morphological Study

Bone can be described as a highly ordered composite of type I collagen integrated with an inorganic mineral phase. In vitro models of bone mineralization using collagenous substrates have been reported in the literature. This study reports an in vitro system of mineralized reconstituted collagen fibers, with aligned fibrillar substructure. The collagen fibers were mineralized in a double diffusion chamber saturated with respect to calcium and phosphate. The morphology and ultrastructure of the mineral precipitate were evaluated as a function of the pH of the incubating media. Brushite crystal was observed at acidic pH. Large rectangular crystals formed at pH 5.15 and appear to associate with the collagen fibers. At neutral and alkaline pHs, hydroxyapatite crystals were observed in association with the collagen fibers . Spherical aggregates of hydroxyapatite crystals were seen at neutral and alkaline pHs, but these structures were reduced in size when formed on collagen at alkaline pH. On close examination these spherical structures were found to be hollow when viewed in cross section. The crystals precipitated within the interior of the collagen fiber at neutral and alkaline pHs were comparable in both size and shape to crystals observed in mineralized turkey tendon and skeletal tissues. These preliminary observations indicate that with further refinement the reconstituted collagen fibers may prove useful in model systems for the study of collagen mediated mineralization in vitro. In addition, mineralization of collagenous matrices may lead to the development of biomaterials for bone repair and replacement

Polaritons and Pairing Phenomena in Bose--Hubbard Mixtures

Motivated by recent experiments on cold atomic gases in ultra high finesse optical cavities, we consider the problem of a two-band Bose--Hubbard model coupled to quantum light. Photoexcitation promotes carriers between the bands and we study the non-trivial interplay between Mott insulating behavior and superfluidity. The model displays a global U(1) X U(1) symmetry which supports the coexistence of Mott insulating and superfluid phases, and yields a rich phase diagram with multicritical points. This symmetry property is shared by several other problems of current experimental interest, including two-component Bose gases in optical lattices, and the bosonic BEC-BCS crossover problem for atom-molecule mixtures induced by a Feshbach resonance. We corroborate our findings by numerical simulations.Comment: 4 pages, 3 figure

Calculation of Densities of States and Spectral Functions by Chebyshev Recursion and Maximum Entropy

We present an efficient algorithm for calculating spectral properties of large sparse Hamiltonian matrices such as densities of states and spectral functions. The combination of Chebyshev recursion and maximum entropy achieves high energy resolution without significant roundoff error, machine precision or numerical instability limitations. If controlled statistical or systematic errors are acceptable, cpu and memory requirements scale linearly in the number of states. The inference of spectral properties from moments is much better conditioned for Chebyshev moments than for power moments. We adapt concepts from the kernel polynomial approximation, a linear Chebyshev approximation with optimized Gibbs damping, to control the accuracy of Fourier integrals of positive non-analytic functions. We compare the performance of kernel polynomial and maximum entropy algorithms for an electronic structure example.Comment: 8 pages RevTex, 3 postscript figure

Self-Modification of Policy and Utility Function in Rational Agents

Any agent that is part of the environment it interacts with and has versatile actuators (such as arms and fingers), will in principle have the ability to self-modify -- for example by changing its own source code. As we continue to create more and more intelligent agents, chances increase that they will learn about this ability. The question is: will they want to use it? For example, highly intelligent systems may find ways to change their goals to something more easily achievable, thereby `escaping' the control of their designers. In an important paper, Omohundro (2008) argued that goal preservation is a fundamental drive of any intelligent system, since a goal is more likely to be achieved if future versions of the agent strive towards the same goal. In this paper, we formalise this argument in general reinforcement learning, and explore situations where it fails. Our conclusion is that the self-modification possibility is harmless if and only if the value function of the agent anticipates the consequences of self-modifications and use the current utility function when evaluating the future.Comment: Artificial General Intelligence (AGI) 201

Order N Monte Carlo Algorithm for Fermion Systems Coupled with Fluctuating Adiabatical Fields

An improved algorithm is proposed for Monte Carlo methods to study fermion systems interacting with adiabatical fields. To obtain a weight for each Monte Carlo sample with a fixed configuration of adiabatical fields, a series expansion using Chebyshev polynomials is applied. By introducing truncations of matrix operations in a systematic and controlled way, it is shown that the cpu time is reduced from O(N^3) to O(N) where N is the system size. Benchmark results show that the implementation of the algorithm makes it possible to perform systematic investigations of critical phenomena using system-size scalings even for an electronic model in three dimensions, within a realistic cpu timescale.Comment: 9 pages with 4 fig

Cardiac output during CPR: a comparison of two methods

Simultaneous Fick and saline dilution methods were compared for measuring cardiac output during experimental cardiac arrest and resuscitation in anesthetized dogs. During cardiopulmonary resuscitation (CPR) cardiac output averaged 53 ± 30 ml/min-kg (42% of pre-arrest values). Values obtained using the Fick vs. saline methods were highly correlated (r = 0.96), and were not statistically different (t = 1.47, df = 16)

Experimentally induced root mortality increased nitrous oxide emission from tropical forest soils

We conducted an experiment on sand and clay tropical forest soils to test the short‐term effect of root mortality on the soil‐atmosphere flux of nitrous oxide, nitric oxide, methane, and carbon dioxide. We induced root mortality by isolating blocks of land to 1 m using trenching and root exclusion screening. Gas fluxes were measured weekly for ten weeks following the trenching treatment. For nitrous oxide there was a highly significant increase in soil‐atmosphere flux over the ten weeks following treatment for trenched plots compared to control plots. N2O flux averaged 37.5 and 18.5 ng N cm−2 h−1 from clay trenched and control plots and 4.7 and 1.5 ng N cm−2 h−1 from sand trenched and control plots. In contrast, there was no effect for soil‐atmosphere flux of nitric oxide, carbon dioxide, or methane

Fine root dynamics and trace gas fluxes in two lowland tropical forest soils

Fine root dynamics have the potential to contribute significantly to ecosystem-scale biogeochemical cycling, including the production and emission of greenhouse gases. This is particularly true in tropical forests which are often characterized as having large fine root biomass and rapid rates of root production and decomposition. We examined patterns in fine root dynamics on two soil types in a lowland moist Amazonian forest, and determined the effect of root decay on rates of C and N trace gas fluxes. Root production averaged 229 ( 35) and 153 ( 27) gm 2 yr 1 for years 1 and 2 of the study, respectively, and did not vary significantly with soil texture. Root decay was sensitive to soil texture with faster rates in the clay soil (k5 0.96 year 1) than in the sandy loam soil (k5 0.61 year 1),leading to greater standing stocks of dead roots in the sandy loam. Rates of nitrous oxide (N2O) emissions were significantly greater in the clay soil (13 1ngNcm 2 h 1) than in the sandy loam (1.4 0.2 ngNcm 2 h 1). Root mortality and decay following trenching doubled rates of N2O emissions in the clay and tripled them in sandy loam over a 1-year period. Trenching also increased nitric oxide fluxes, which were greater in the sandy loam than in the clay. We used trenching (clay only) and a mass balance approach to estimate the root contribution to soil respiration. In clay soil root respiration was 264–380 gCm 2 yr 1, accounting for 24% to 35% of the total soil CO2 efflux. Estimates were similar using both approaches. In sandy loam, root respiration rates were slightly higher and more variable (521 206 gCm2 yr 1) and contributed 35% of the total soil respiration. Our results show that soil heterotrophs strongly dominate soil respiration in this forest, regardless of soil texture. Our results also suggest that fine root mortality and decomposition associated with disturbance and land-use change can contribute significantly to increased rates of nitrogen trace gas emissions